1. Field of the Invention
The present invention relates to a system for automatically generating a seamless and continuous developed still image by generating a developed diagram in the circumferential direction of a tube based on one frame of a video image of an inner wall of a tubular object photographed while moving in the axial direction of the tubular object, and then, stitching the developed diagrams in the longitudinal direction (i.e., on a central axis) of the tube by mosaic processing in order to examine the inner wall of the tubular object.
2. Description of the Related Art
The inner wall of a tunnel, which is one kind of tubular objects (for example, a building structure such as a tunnel, a water supplying pipe and tubular organs of human beings or animals such as esophagus, stomach, large intestine, small intestine, rectum, blood vessel, urethra, ureter, urinary bladder and trachea are regarded as the tubular objects in wide sense) has been conventionally examined by direct observation by a person in the site and sketch accompanied therewith or the like. However, this has been a hard, dirty and dangerous work, requiring much manpower and much time, and further, it has been difficult to accurately specify an abnormal portion.
Therefore, in order to relieve a person of such work in the tunnel, there have been proposed several examining methods based on an image photographed by a camera.
For example, in a method disclosed in Japanese Patent Application Laid-open No. 6-42300, a curved mirror is used, a wall surface of a tunnel reflected on the curved mirror is photographed by a camera, obtained images in a tunnel circumferential direction are arranged according to a camera movement distance, thereby forming developed images of the tunnel. Here, it is necessary to design the shape of the curved mirror or optically design a lens in the camera according to the cross-sectional shape of the tunnel, thereby raising a problem of a complicated design work since the cross-sectional shape of the tunnel is not constant. Moreover, the tunnel in the circumferential direction at a certain position in an advancing direction is reflected on the curved mirror, and then, it is photographed by a single camera, thereby inducing a disadvantage of a low resolution of the image.
In the meantime, such a method has been proposed that the image reflected on the curved mirror is not photographed, but a line sensor camera is fixed to a rotary device installed inside of a tunnel, an image of a wall surface of the tunnel is obtained by scanning the tunnel wall surface in the circumferential direction thereof, and the images obtained by repeating the photographing in the same manner after movement by a predetermined distance in a tunnel axial direction (i.e., a movement direction) are combined with each other, to form a continuous developed image in the tunnel axial direction in order to directly photograph a cross-sectional shape. Such a method has been proposed in, for example, Japanese Patent Application Laid-open No. 11-294065 or 11-81879, which raises problems that a special and expensive camera is used, and further, that a photographing time becomes long.
Moreover, the tunnel is not photographed several times at predetermined angles by rotating the single camera in the axial direction, as described above, but there has been proposed a method for setting four line sensor cameras at substantially the center of the tunnel in different directions, so as to photograph the entire circumference of a wall surface of the tunnel at four sections, as disclosed in, for example, Japanese Patent Application Laid-open No. 2001-43353. However, this method requires the four expensive cameras having a wide angle, thereby raising a problem of marked laboriousness of adjustment of photographing conditions of the four cameras such as the position, direction and moving speed of the camera.
In order to solve the above-described problems, there has been proposed a method for creating a circumferentially developed diagram of each of frames of an image (photographed in a doughnut-like form) of the inner wall of a tunnel photographed by a video camera, which moves in the axial direction of the tunnel, and combining a plurality of the developed diagrams in an advancing direction (i.e., the axial direction), so as to obtain a continuous developed image in the axial direction of the tunnel, as disclosed in Japanese Patent Application Laid-open No. 11-66316.
However, such a method raises the following problems:
(1) Complicated procedures such as control of the attitude and capture of positional information of the camera are required.
This is because a computer regards all of video information as numerical values. It is necessary to accurately feedback the attitude or positional information of the camera causing a factor of fluctuations of the image as the numerical values in processing a continuous video image with a time change in high quality.
Therefore, data output from a sensor must be simultaneously acquired by mounting a sensor (e.g., a gravity sensor, a gyroscopic sensor or the like) capable of acquiring the attitude information on the side of the camera, thereby requiring a special camera device. Furthermore, a sensor for acquiring the positional information or traveling distance data (i.e., a cable sending quantity) is mounted, so that the information or data is superimposed on the image by a character generator. It is necessary to recognize the superimposed character in order to acquire the positional information or traveling distance data. However, character recognizing processing requires immense labor, and further, there might be a danger of a data loss caused by erroneous recognition.
(2) Since the frames are merely connected to each other, no seamless mosaic image can be obtained.
(3) Since the frames are merely connected to each other, the lateral (i.e., longitudinal) size of the image between the frames cannot become constant if a photographing movement speed is not constant, thereby causing the distortion of the image.
(4) Since a distorted portion and a non-distorted portion are superimposed on the image at the same portion if mosaic processing is performed by the use of all of the frames, it is necessary to classify images which are used or not used in the developed diagram by eyes of a person. Therefore, the mosaic processing (i.e., image stitching/image alignment) cannot be automatically performed, thereby raising a problem that it takes a long time.
Moreover, when a tubular organ as one kind of tubular objects is diagnosed by an endscope, an image of an affected part and its vicinities in an inner wall are frequently photographed by a small-sized CCD camera on a medical worksite. The photographed image is, in some cases, displayed on a display as a video image, or a part of a video frame is, in other cases, printed as a still image for use. In the former case, there are drawbacks that it is inconvenient to store the image together with a chart or the like, and further, that it is impossible to accurately grasp the positional relationship of the affected part with the inlet of an organ. In the latter case, since the sequential images are taken out as several pieces of partial photographs, it is difficult to grasp the entire image, and further, high-grade expert knowledge or experience is required to determine that the part corresponds to which part of the entire image. Moreover, since the number of photographs to be printed is limited, the photographs near the affected part are inevitably main, thereby causing a possibility of overlooking metastasis of cancer. In both of the former and latter cases, since the endscope CCD camera performs photographing while moving in the advancing direction (i.e., the axial direction) of the tubular organ, an obtained image becomes a doughnut-like image just as one obtained by photographing a tunnel. In order to photograph the state of the inner wall in detail, the CCD-camera needs to be rotated in the circumferential direction of the tube, followed by photographing. However, such operation is not actually achieved in a human body in consideration of a pain of a patient.
In such a case, although it is very convenient to display or print the photographed video image (i.e., the doughnut-like image) of the inner wall of the tubular organ as a developed continuous still image, for now, there exists no such system for automatically converting the video image of the inner wall of the tubular organ into the continuous developed still image.